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Introduction 


The purpose of the School Panel Review Process is to assist the Commissioner of Education in 
determining whether State intervention is needed to guide improvement efforts in schools where 
students' MCAS performance is critically low and no trend toward improved student 
performance is evident from MCAS data. John Marshall Elementary School met this criterion for 
mathematics and was one of sixteen schools selected for panel review in winter, 2004. The panel 
review was conducted on January 26-27, 2004. 


The review panel's charge was to analyze data and written information on the school's 
performance and improvement efforts, visit the school, and meet with school and district 
officials in order to advise the Commissioner on the answers to the following two key questions: 


1. Does the school have a sound plan for improving student performance? 
2. Are the conditions in place for the successful implementation of the school's 
improvement plan(s)? 


The panel's responses to the two key questions that defined the scope of its review are included 
in this report. These findings and conclusions are the product of the panel's analysis, discussion, 
and observation, based on the evidence available to it. A list of panel members who participated 
in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided 
in Appendix B. 


The panel's findings and conclusions on the two key questions will be forwarded to the 
Commissioner of Education for consideration, together with school performance data, in 
determining whether John Marshall Elementary School is deemed under-performing. The panel 
was not asked to formulate a sound plan for school improvement where such a plan does not 
presently exist or to recommend a course of action to create the conditions for successful 
implementation of sound improvement strategies where such conditions at present do not appear 
to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing 
school, occurs at the next stage of the school review process. 


John Marshall Elementary School Profile 

Enrollment 

The John Marshall School is one of 83 elementary schools in Boston. The school serves students 
in kindergarten through grade five. Preliminary results show 688 students enrolled at the school 


as of October 1, 2003. Enrollment between 2001 and 2003 ranged from 775 and 815 students. 
The majority (63 percent) of students attending the Marshall School over the last four years have 
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been Black, followed by Hispanics, who composed 33 percent of the total student body. White 
and Asians each account for two percent of students at the Marshall School. 


Over 80 percent of students at the Marshall School come from low-income families. Over the 
last four years, an average of 30 percent of students have been non-native English speakers; over 
20 percent have been Limited English Proficient students. Fifteen percent of students enrolled 
last year and this year receive special education services. 


Since 2000, attendance at the Marshall School has been steadily declining. In 2003, the 
attendance rate was 94.3 percent, compared with 94.9 in 2000. The average number of days 
students were absent during this period was nine. Retention rates at this school have steadily 
declined over the last four years, from eight percent in 2000 to 3.8 percent in 2003. Last year, 
out-of-school suspensions climbed to 8.1 percent from 0.4 percent in 2002. No in-school 
suspensions were reported between 2000 and 2003. 


Staffing 


In its Staff Data Report submitted this academic year, the Marshall school reported having a staff 
of 62 that includes four administrators, one guidance counselor, one nurse, six support people, 
and 54 teachers. Forty percent of the staff has been there for fewer than five years. All but three 
teachers are licensed in the subjects they teach. 


MCAS Overview 


Students at the John Marshall School are assessed in grades 3 and 4 in English language arts 
(ELA), and in mathematics in grade 4. In mid-Cycle III (2003), the school failed to make 
Adequate Yearly Progress (AYP) in the aggregate and for all its qualifying subgroups ELA.' In 
the last five AYP determination cycles, the school received a positive determination twice. In 
mathematics in 2003, the school also failed to make AYP in the aggregate and for all its 
subgroups, except for African-American/Black. In the last five years, the school has not made 
AYP in this content area. 


Student Performance in English Language Arts 
GRADE 3 
Regular Education 
Although the distribution of regular students’ performance at the Marshall School at the grade 3 


level shows a slight improvement in 2003, there are no trends of sustained improvement in ELA. 


' In accordance with the federal No Child Left Behind Act passed in 2001, student performance is disaggregated by 
the following subgroups: Limited English Proficient, Special Education, Free/Reduced Lunch, A frican- 
American/Black, Asian or Pacific Islander, Hispanic, Native American, and White. A minimum of 20 students per 
subgroup is required to issue a statistically sound rating or determination of Adequate Yearly Progress (AYP). The 
subgroups meeting the minimum sample size at the Marshall School in 2003 were Limited English Proficient, 
Special Education, African-American/Black and Free/Reduced lunch and Hispanic. 
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In 2001 when the test was first administered, 31 percent of students were proficient; 56 percent 
performed at the Needs Improvement level; and 14 percent at Warning. In 2002, the proportion 
of proficient students fell to 28 percent; 62 percent were in need of improvement, while 10 

percent scored at the Warning level. In 2003, there were 37 percent of all assessed third graders 
performed at the Proficient level, 50 percent at Needs Improvement and 13 percent at Warning. 


Special Education 


The performance of special education students in grade 3 in ELA has remained constant over the 
last three administrations. Six percent of students have performed at the Proficient level, about 
42 percent at Needs Improvement, and 50 percent at Warning. 


Limited English Proficient 


In 2003, the performance of Limited English Proficient (LEP) students saw a sharp reversal of 
the trends from the two previous years. In 2001, 65 percent of assessed LEP students were 
proficient, and the remaining 35 percent performed at the Needs Improvement level. In 2002, the 
percentage of proficient LEP students rose to 73 percent, with the proportion of those at the 
Needs Improvement level declining to 27 percent. In 2003, there were no proficient LEP 
students. Forty-six percent scored at the Needs Improvement level while the majority (54 %) was 
at Warning. | 


GRADE 4 


In 2001, the performance of regular education students at the grade 4 level made some strong 
gains; since then, however, performance at this grade 4 level has been static. In 2000, one 
percent of students tested at the grade 4 level was proficient; 84 percent scored at the Needs 
Improvement level and 15 percent at Warning. In 2001, the percentage of proficient students rose 
to 18 percent. Sixty-four percent of students were at the Needs Improvement level, and 18 
percent at Warning. In 2002 and 2003, the proportion of proficient students has remained 
constant at 18 percent. In 2002, sixty-nine percent of regular education students were at the 
Needs Improvement level and 14 percent at Warning. In 2003, 62 percent were in need of 
improvement with the remaining 20 percent scoring in the Warning category. 


Special Education 


The performance of special education students at the Marshall School over the last four years has 
been marked by high percentages of students performing at the Warning level, and very few, if 
any, reaching proficiency. Except for 2001 and 2003 when eight and four percent of students 
respectively performed at Proficient, there have been no proficient special education students at 
the grade 4 level. Over the last four years, on average, 30 percent of students have scored at the 
Needs Improvement level and 70 percent at Warning. 


Limited English Proficient 
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The performance of LEP students has varied widely over the last four years. In 2000, there were 
no proficient LEP students in grade 4 in ELA. Seventy-three percent performed at the Needs 
Improvement level and 27 percent at Warning. In 2001, 23 percent of those assessed were 
proficient, 54 percent in need of improvement, and 23 percent scored in the Warning category. In 
2002, 93 percent of LEP students’ performance was in the Needs Improvement category and 
seven percent in Warning. In the most recent administration of the math test, 33 percent 
performed at Needs Improvement and 67 percent at Warning. 


Student Performance in Mathematics 
GRADE 4 
Regular Education 


The performance of regular education students in mathematics at the Marshall School has shown 
no improvement since 2000. In 2000, seven percent of students tested in this content area were 
proficient; 68 percent in need of improvement, and 25 percent performed at Warning. In 2001, 
four percent of students score Proficient and Advanced. The proportion of those at the Needs 
Improvement level fell to 58 percent as those in Warning rose to 38 percent. In 2002, the 
percentage of proficient regular education students decreased to four percent, as did those in 
need of improvement to 46 percent. That year, 50 percent of students performed at the Warning 
level. At the last administration, eight percent of students were found proficient and advanced, 
51 percent in need of improvement, and 40 percent scored at the Warning level. 


Special Education 


The performance of grade 4 special education students in mathematics has been static for the last 
four years. Except for 2000 and 2003 when two and three percent of students, respectively, were 
found proficient there has not been other proficient special education students in mathematics. 
Between 2000 and 2003, each year on average one-third of students have performed at the Needs 
Improvement level while the remaining two-thirds of scores have been at the Warning level. 


Limited English Proficient 


While the performance of LEP students in mathematics has varied over the last four years, there 
is no evidence of improvement. In 2000, 71 percent of all LEP students scored at the Needs 
Improvement level, and 29 percent at Warning. In 2001, 13 percent performed at the Advanced 
level of performance; 63 percent were in need of improvement, while 25 percent scored at 
Warning. In 2002, LEP students’ performance as evenly divided between the Needs 
Improvement and Warning categories. Last year, 54 percent of scores were at the Needs 
Improvement level and 46 in Warning. 
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PANEL RESPONSES TO THE KEY QUESTIONS 


KEY QUESTION 1: DOES THE SCHOOL HAVE A SOUND PLAN FOR IMPROVING 
STUDENT PERFORMANCE? 


Yes. The John Marshall Elementary School has produced a Whole School Improvement Plan 
(WSIP) that is clearly focused on several key instructional priorities in English language arts 
(ELA) and mathematics. The school has analyzed data from various sources to examine student 
achievement, including both strengths and gaps, and instructional needs going forward. The 
process involved the collaboration of the entire staff in the determination of root causes and the 
development of strategies to address resulting deficits. 


A. Has the school analyzed appropriate data and program information to accurately 
identify the gaps in student performance and determined why those gaps exist? 


Yes. As stated in the plan and confirmed in interviews with staff and the school principal, the 
process of data analysis began in the fall of 2002 when the school’s Instructional Leadership 
Team (ILT), which had received three days of training in data analysis the previous summer, 
commenced the review of both MCAS and Boston Public Schools (BPS) formative assessment 
data for the entire school as well as disaggregated subgroup results. The process included the 
work of grade level teams, which analyzed test items for areas of weakness, as well as the efforts 
of the entire staff working together on root cause analysis. The ILT also examined the school’s 
instructional programs in mathematics and ELA to determine and address weaknesses in 
implementation. 


The school reviewed MCAS scores in mathematics for the years 2000, 2001 and 2002 for the 
fourth grade. Additionally, BPS formative math assessments for grades 1-5 were examined. This 
review, aligned with standards, showed that student deficits were greatest in attainment of 
proficiency in the areas of number sense and open response questions, and led to the articulation 
of the school’s math priorities to (1) build students’ understanding of number sense and problem 
solving strategies; and (2) build students’ ability to communicate their mathematical thinking 
using words, numbers, and/or pictures in the context of open response questions. 


For ELA, scores for 2001 and 2002 for grades 3 and 4 were analyzed. BPS formative reading 
and writing assessments were determined to be below the district’s grade level benchmarks. The 
data found weaknesses in proficiency in vocabulary and reading comprehension skills as well as 
those related to writing with organization, focus, and adequate topic development. The school 
then determined that its ELA priorities are to (1) strengthen reading comprehension for all 
students across all genres with an emphasis on vocabulary; and (2) build the writing skills of all 
students with a strong focus on vocabulary and topic development. 


When results were disaggregated by subgroups, the school noted similar weaknesses for all 
students and formulated the same priorities. It is in the area of instructional strategies that 
adaptations and accommodations have been enumerated for special education students and 
English language learners in the WSIP, and in the newly completed appendix to the WSIP that 
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was provided to the panelists during the visit. There was a severe decline in the 2003 test scores 
of ELL learners who previously had been tested in Spanish. MCAS regulations required eleven 
more ELL students to be tested in English for the 2003 administration (two more on grade 3 
ELA, five more on grade 4 ELA, and 4 more on grade 5 mathematics). 


The ILT also examined the school’s instructional programs and practices to determine the extent 
and success of implementation. The school’s math program, TERC Investigations, was only in its 
first year of full adoption when the plan was developed last year. A number of weaknesses were 
identified such as inconsistency in teachers’ use of the BPS Scope and Pacing Guide, inadequate 
engagement of students in considering and analyzing solutions, and lack of consistency in 
requiring students to use appropriate mathematical terminology and notation. Other identified 
deficits were the lack of training for some teachers in /nvestigations and the lack of intensity and 
consistency from a BPS coach in mathematics. A coach is only provided by the district twice a 
month, and the coach has been changed four times during the year. 


For Writer’s Workshop, the district’s instructional program for writing, the ILT identified 
implementation weaknesses including lack of uniformity in keeping notes on conferences with 
students to inform instruction; a need to implement genre-based units of study in all classes; and 
inconsistent posting of learning objectives in all classrooms. For Readers’ Workshop, lack of 
uniformity in conferring with students, inconsistency in use of texts reflecting various genres, 
and the need to incorporate word study and vocabulary were cited as instructional weaknesses. 


The root cause analysis in the WSIP addressed the gaps that exist due to inconsistent delivery of 
the instructional program; limited use of assessment to inform instruction; limited practice 
afforded to students to apply problem-solving strategies; inappropriate utilization of instructional 
time; lack of teacher questioning to encourage students to reflect and verbalize; insufficient 
emphasis on acquisition and use of new vocabulary; and providing students with only minimal 
experience with a wide variety of genres. 


The school’s curricula are guided by Boston Public Schools Standards, which are reportedly 
aligned with the Massachusetts Curriculum Frameworks. Teachers now have access to, and are 
supposed to be teaching from curriculum guides available on the district’s MyBPS web site. 
Schools were also directed this year to create curriculum maps for ELA, which would lay out a 
scope and sequence already available in pacing guides customized by the district from guides 
provided by the TERC Jnvestigations math curriculum. 


In addition, certain programs are based in the school according to district needs and the size of 
available facilities. This year, the John Marshall Elementary School is a new Sheltered English 
Instruction Center for Spanish speaking students. A substantially separate language-based class 
~ houses students who are assigned from other schools beginning in fourth grade by the special 
education department. There is also a Learning Adaptive Behavior (LAB) cluster behavior 
modification program that serves students in all grades. Based on high Stanford Achievement 
Test scores, approximately ten fourth grade students leave the school annually to attend an 
advanced work program in another building. As a result of these district assignment practices, 
Marshall Elementary serves special populations that may present instructional challenges. 


February 17, 2004 Massachusetts Department of Education 


d+a89 7 . (ooo? yatrenata tare ndiok eo THEW? Froqest 


> 


og feed ENOL aff (ap itantagooeitieaeiaal pane eer aaron dit 
ovois boriwpat at rveh chGiemery. AM’ Lag ra hutagt creel best ylaneiveny oy 2 

€ shag wp sida owas nots naimitie © POS ol tet Malia 3 ft hytest oot by wish ned 
fashininedtnen 2 ue » sare bis ATS be pling AG avon : 


. 


mei ult cine of asst ig ee Sitie peo hee sie ea 3 bowloa at! bavi ¢ 
W! > Vinee ZEW ROA ake ft te SRST Le EAN 2 ood ae att pee area: 
sis esaearlcew 1) 190 A. amo" tage egy vil ewer Pinicy nis ey noobs ite te 
ony ‘Subisel coraateitosnl Getlk 
th vores Me Zod bad werondles oui lene ldisthinsbieand a. Pinabiite Yo 
often elt .coblelin Sate hela Tastes My aitsrettts rex sehaenga ogy ? atrsk 
brea veitegeety? Leon! eal bem} ORS ety r Ze ent sitrow inl jainiis! to foal ort 99 
& sormiontuib oli vd seein rn 4 2) dost) ty 2obenligig at dosoe STE 6 not y 
Mio oto iret sole) vai baer sed ann Pipes ern 


‘ ; 4 ST eete ue i ay Dele: oo) Dy od 
nuh 4 ahs We). on i oAF '? XY ; aq 4 - r 4) aid re 


ys 


onion? TA ond et hiiw vot virtagrs, ienotiodend eYorneih ant god i ae 

iY arty TeIS Ny SOR a iG oN 51h TAOS aan aI oy ainialoun ae Le ad notte 
has 7esecelo in ef wot) Bui DP s-sngee rote] ical of Heres seo wm 
id sit) arlewe sy) “erabReN had esaronen ian! eovatuo io simian! 30 gauze In 
ngs arengy woippetta) icaae sevpemelanodnt doatate rif sores 
onevininaw bandertiedd es batis sow ‘cies tae wld thee ne ecnnon} of 85 


halt 


, ce VIM SAD Yep ites (ig fT i era5 ERY ie te atin qtr f Wiebe, a In| or " pint it il (ia 


‘ ! ee) 7 ry & 
659i! tata PUR TLSE II FOOTE OP Sah ie ORR TO) aaeeh BASTIEN ANG oz 
lartonaiwer: if) noes aU |) hire) Sahette withvlog-t ato th ER uly i OF elegy 
lraitiuent anttinesy bac toile: of aire mje 12 SE MWOGHS Gi ahaa emtrte barca 


leminec view bw -cinstils cothivery bits vinta wire 1 ma Ba eosinpan A 
7) env | ap: ie, Yo Yabiney sive a ‘s 


he ve 


via tiaqet ne tio yr upiglintie oh arise cei ge 20d ad sig ae whuohnye 2 
“ones hin peas 756 aes! yon rer noe: eae it betas AMT eR: Bite. “ent aett 
oi dow a hee tolnisiht cdi abdaliov sy: 20h nepissoinad ei grysseoh oda 

§ no hci hi) cw Sirius Ae Pap irin tt r Sse! radsr> ie 30.0 amie ail ist BSE C ai SEs 
ssbiug mei tatvelh orl) bsshnelgny. gabe an “fae pet sidatinye ¢birta sotoipadt 


— eee rs ey? a a ‘eT a7 
festuopmeeiteer iimaticie SHAT on raed be ) 
io ota orf (tie deeeriori of pidiioonk lost. Sita Pete as Bit tacrl maths 


dalen’ pemtlatt win ga lomisPyeor ell dae stitak a at fasy wid) cuits 
g26!9 bipend Maroc { singcese yileriaredie A sterwtina af eieinede i maine ) no 
nisi ol yd hain ff i ti, sarin ob aiiedoe worl sar BONBiarck 514 Orive first 2133 
vedo) saeco GEA) tolvure®t witb seit s vale 2hecalT seeranaceb all 
SisveulndA noe oo rise ssi ne 497 14 i an Harty & itsiy Mile Bov Nil? mor roulgy til 
ie bnelia cl yen feng a ON! o¥io\ aiiebula ofa hel 1S! “foie Gys Se 
eo 7ouny Insaiyiass tohind oewll totlnen a aA aerblind witons os mmeoy AoW bs: 


reanslleds ishiaffusdeay vesrey yar dec ahpiset hei “uae sovase Visitomald tier 


ieowbd Vo in-ear atsupioeegh SOL Ft vente 


Panel Review Report, Winter ‘04 John Marshall Elementary School page 7 


The school made schedule adjustments for all students to accommodate the district’s mandate for 
increased mathematics instruction. For K-5, one hour each day is devoted to the fast-paced 
Investigations program with an additional ten minutes to review daily math routines. In grades 3- 
5 there is another 30 minutes daily of AYP math to focus on opportunities to answer multiple 
choice, short answer, and open response questions similar to those found on MCAS. These 
opportunities to provide additional practice and review basic operations help to address teachers’ 
concems that the /nvestigations program moves too quickly for the school’s students. 


In the staff report, the team noted that most teachers have a number of years of experience and 
are teaching within their areas of certification. Nine of 62 staff members have three years or less 
of experience in education. Seven had no certification indicated. In the staff survey, of teachers 
generally reported that they know what they are expected to do to fulfill the school improvement 
plan to improve student performance; only one responded “unsure.” Teachers repeatedly cited 
the plan’s priorities such as vocabulary development, solving word problems in math, and 
improving reading comprehension. 


Although the current WSIP does not look back at previous plans to determine how successful 
they were, it is clear from looking at the previous plans that the principal and the ILT are familiar 
with the planning process and the analysis of data. Over time, the BPS has provided training in 
this process and this is reflected in the level of detail apparent in the current plan and its 
accompanying appendix. In particular, the 2003-2004 WSIP appears to be better focused than 
previous plans, as evidenced by the emphasis on multi-step problem solving and vocabulary and 
comprehension development, root cause analysis and clearly defined improvement strategies, 
and thus affords a higher likelihood of successful implementation. The district also encouraged 
schools to look beyond the Six Essentials format of the previous WSIP template and take a closer 
look at causal factors implied in the data 


One set of data that the school did not analyze or address in the plan concerned attendance. From 
the data provided by the Massachusetts Department of Education for the Panel Review, the team 
noted that the school’s attendance rates have dropped from 91.7 percent in 2001 to 87.3 percent 
in 2003. However, this information has been corrected to reflect an actual attendance rate of 94.3 
percent, little changed from previous years. In view of this correction, overall attendance is no 
longer an issue that merits attention in the SIP. 


In 2003, 23.4 percent of students were identified as chronically absent (defined as absent more 
than 10 percent of days in membership). The average number of days absent declined, however, 
from 14.9 in 2001 to 9.4 in 2003. The superintendent stated that attendance is a district-wide 
issue and noted that as a district, Boston did not make AYP because of attendance. The school 
also did not address strategies for its approximately 25 percent in/out turnover rate, as reported 
by the principal, with a plan to acclimate new students promptly. 
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B. Does the plan set out specific improvement objectives that are grounded in the school’s 
analysis of the reasons for poor student performance? 


Yes. The plan delineates specific collaboratively determined goals tied to improved student 
performance on the MCAS. Although these goals are ambitious, they are required to meet AYP 
and are specific and reasonable in terms of the school’s analysis. For grade 4 math, this includes 
goals for English language learners as well as special education students. For grade 4 ELA, there 
is a goal for special education students. For grade 3, there are no disaggregated goals and the 
goals specified are for all students. In the appendix to the WSIP, BPS outcome benchmarks are 
tied to each of the school’s math and ELA instructional priorities and will be measured at 
varying intervals throughout the year. Formative assessments to be used to measure the student 
performance objectives in ELA include DRA, SRI and writing prompts. Formative math 
assessments include MCAS-like multi step problems and open response questions. The John 
Marshall Elementary School is piloting the BPS benchmarks for local assessments (formative) in 
grades three and four as part of the Boston Plan for Excellence. These are being designed to 
more closely parallel MCAS type questions and work in ELA. 


In focus groups and interviews, as well as in the staff survey, teachers demonstrated that they are 
able to articulate student needs and the goals of the plan. The observations conducted by the 
Panel Review Team showed that staff is actively engaged in addressing these needs and fulfilling 
the goals of the plan in the classroom. 


A Classroom Observation Checklist has been developed for each priority in ELA and 
mathematics for teachers and administrators to determine that the plan is on target for 
implementation. The checklist specifies strategies for teachers of regular education students as 
well as the language-based, LAB and ELL classes. Some examples: 


e At least once during a Read Aloud session, LAB teachers will discuss and 
reference genre charts. 

e SEI classroom work will reflect accurate use of graphic organizers. 

e At least once during math class teachers will model questions to ask before 
starting a problem. 

e At least once a week, teachers will plan a lesson using a multi-step or MCAS 
open response problem. 


As a result of these checklists, the plan becomes not only a blueprint for student achievement, 
but one for tgacher improvement as well, with implementation monitored by administration. 
Teachers are also able to monitor their own work with improvement rubrics and a self-reflection 
component that is to be completed in June. 


C. In order to accomplish each improvement objective, does the plan specify strategies 
which appear likely to lead to improved student results? 


Yes. The WSIP for the John Marshall Elementary School specifies a number of strategies tied to 
each instructional priority that are likely to lead to improved student results if they are diligently 
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implemented. The programs and practices in current use in the school and the BPS, such as 
TERC Investigations, and Readers' and Writers' Workshop, are all well-known programs that 
have been successfully adopted in other places. 


In interviews, focus groups and classroom observations, the team determined that math and 
literacy strategies are now in place to implement the plan. Plentiful professional development has 
been made available to staff and many have participated over the past several years. Despite the 
fact that this year teachers are involved in a district wide job action and are working to rule, 
several are currently engaged in the professional improvement opportunities the district offers. 
During the day, there are regular cycles of literacy and math coaching that take place at the 
school. Although the district-provided math coach is only in the building twice a month, 
collaborative coaching and learning (CCL) activities also provide opportunities for teachers to 
improve instruction. As teachers take turns observing each other and providing feedback, they 
are able to reflect on the strategies in the WSIP. 


One of the promising interventions has been the designation of a Math Leadership Team (MLT). 
This group meets regularly to advance the school’s math program. It is comprised of one 
member from each of the school’s grade level instructional teams. Since the school is in 
corrective action because of lack of AYP in math, the establishment of this group has been 
particularly timely and can be a key to the school’s turnaround. 


D. Are the school’s written improvement planning document (s) clear and specific enough 
to guide their implementation of planned improvement initiatives? 


Yes. Teachers report that the plan has indeed changed the focus of their work with students. In 

interviews, several teachers indicated that WSIP priorities are considered in planning. In focus 

groups, teachers stated that the strategies in the plan were clear and would lead to improvement 
with implementation. They also declared that the plan contributes to consistency from grade to 

grade and expressed appreciation of its focus and specificity. 


The administrative team has created an Improving Students’ Performance Tool Kit and provided 
it to each teacher. This extensive and impressive document includes the WSIP and appendix; the 
key elements of effective lesson plans including planning and preparation, connections/building 
background, learning strategies, review and assessment; BPS curriculum information for ELA 
and math including learning standards, assessments and information on workshop instruction for 
ELA and /nvestigations for math. The math section is particularly detailed including pertinent 
unit vocabulary, suggested teacher questions, and the pacing guide. Sections for science and 
social studies instruction are also included. 


The team observed numerous instances of implementation of planned improvement activities 
including: 


e Students showed classmates how they solved specific problems. 


e Students showed how they solved a problem in two different ways and explained how 
both ways would give the same correct answer. 
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e Students have math journals filled with examples of how they solved problems. 
Examples of this kind of writing were also posted on walls. 

e Students used math vocabulary in their explanations. 

e Math and other vocabulary lists were posted in classrooms on word walls. 

e Teachers modeled metacognitive approaches to answering questions. 

e Students, including children in first grade, were observed reading independently and 
writing about what they read. 

e Lesson objectives were articulated by teachers and were posted in classrooms. 

e Teachers’ plan books were detailed and focused on student learning objectives. 


In the Department of Education staff survey, teachers were asked what they are expected to do, 
according the school’s improvement plan, to improve student performance. One teacher’s 
response stated it succinctly and demonstrates the plan’s clarity: 


Build problem solving skills including problems with ...multiple steps. Build writing 
skills and students’ vocabulary and topic development. Strengthen reading 
comprehension across all genres. 


E. Was the School Improvement Plan developed through a process that will support its 
successful implementation? 


Yes. The ILT, which consists of nine teachers, the principal, assistant principal, literacy coach, 
and LAB cluster coordinator, had primary responsibility for the development of the plan. This 
group meets every other Monday. Minutes are kept and available to all teachers. Teachers had an 
opportunity to view the plan in process during professional development and grade level 
meetings. Grade level teams analyzed MCAS data and looked at student responses to questions. 
It was a whole school process that looked for causes for poor student performance and delineated 
plans and strategies for improvement that included benchmarks married to instructional 
priorities, timelines and persons responsible. While the main impetus for change may frequently 
originate with the ILT, the Panel Review Team heard from many teachers that the grade level 
meetings provided a productive source for the content of the final plan. 


The Superintendent claimed that this plan represents an “exemplar” for the district. The school’s 
structure supports successful implementation. Weekly grade level team meetings provide 
opportunities for looking at student work and planning for curriculum implementation and 
Collaborative Coaching and Learning (CCL) sessions. At these meetings, representatives from 
the ILT report on whole school initiatives and concerns. Communication between the UCM 
and grade level teams is a back and forth open process that allows for sharing among teachers in 
the rollout of the plan and its implementation. 


The Panel Review Team, however, did note that the Leadership Report and the staff identified 
professional development needs that were not followed through in the WSIP and implementation 
to date. These needs include strategies for diverse learners including LEP students, alternative 
strategies for students with disabilities, and additional coaching in math. 
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KEY QUESTION 2: ARE THE CONDITIONS IN PLACE FOR THE SUCCESSFUL 
IMPLEMENTATION OF THE IMPROVEMENT PLAN (S)? 


Yes. The Panel Review Team determined that the school’s prospects for effective 
implementation of the School Improvement Plan are facilitated through strong leadership, faculty 
support, and district oversight. The principal is a strong instructional and organizational leader 
who commands utmost respect from the school. She accepts input from the school’s 
Instructional Leadership and School-Based Management Teams as well as classroom teachers. 
These groups have helped clarify the school’s learning priorities and have created the roadmap to 
reach the proposed goals. 


A. Does the school have effective leadership and sound management? 


Yes. The school has a capable principal who is a knowledgeable and effective instructional 
leader with a clear focus on putting children first. A number of structures are in place to assist 
with leadership, management and communication functions including the instructional leadership 
team, grade level teams, a math leadership team (MLT) to work toward full implementation of 
Investigations; and a school-based management team (SBT), which is the school site council that 
meets monthly. This latter group actively participates in programmatic, budget and hiring 
decisions subject to school and district constraints. The school has been structured into five K-5 
families (each with the name of a planet) and located in distinct clusters in the building. Students 
remain in the same family throughout their years at Marshall. The school’s assistant principals 
are each in charge of one or two K-5 families, and perform a number of administrative and 

_ supervisory responsibilities. 


The principal is pro-actively monitoring the /nvestigations program to ensure implementation. 
She participates, along with teachers, in all professional development, sits in on some of their 
grade level meetings, inspects plan books and, according to teachers, establishes non-negotiable 
expectations for their performance. She has mandated peer observation (CCL) of at least one 
lesson each year in math and ELA. She has been designated as a cluster leader for the district to 
mentor other principals and is considered by the superintendent to be one of Boston’s 
outstanding principals. In one focus group, the principal was acknowledged as one of the 
“strongest administrators in the city.” 


Now that the plan has been completed, the ILT is involved in all aspects of school governance, 
has non-evaluative monitoring responsibility and reviews children’s notebooks. Members meet 
in their grade level teams to discuss student work and provide feedback to teachers and act as 
mentors to provide curriculum support. ILT members also participate in the CCL process. The 
ILT initiated the production of the Tool Kit, which, together with the Implementation 
Benchmarks document recently developed as an_appendix to the WSIP, will provide a 
monitoring tool to gauge progress in implementation of the required improvement strategies. 
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Grade level teams meet every Wednesday for 90 minutes to examine student work, develop 
pacing charts, and share and review classroom literature and math units. Agendas for the grade 
level teams may be determined by the ILT, by the administration or by the group itself 
depending on what needs to be accomplished. Each grade level team has a representative on the 
ILT and the MLT who bring relevant information to the table. This process helps to ensure 
standardization of the curriculum in math and ELA. Each SPED resource room teacher works 
with one grade level and meets with grade level teams. Families, however, only break into 
groups about four times a year during school wide professional development, which may limit 
opportunities for vertical articulation of the curriculum, despite the fact that test data has, for the 
past two years, been broken down by family according to the principal. 


The principal has modified schedules to enable teams to meet and have effective common 
planning time and classes to have more instructional time in mathematics. She is aware of the 
school’s math needs and plans to eliminate one assistant principal to add two half-time in-house 
math coaches next year. 


B. Is there evidence that the school’s faculty supports the planned improvement efforts? 


Yes. Teachers agreed in the staff survey, interviews and focus groups that the school’s goals in 
ELA and math are appropriate and necessary. The team observed fourteen teachers and 
interviewed them individually; in addition to the ILT, MLT and SBT, over 30 teachers were 
interviewed in focus groups. The evidence was overwhelming that the staff has embraced the 
planned improvement efforts. Teachers participated in the development of the plan from analysis 
of data to designation of root causes and development of strategies to address them. They agree 
that a critical focus of the plan is to build math and ELA vocabulary. From observation and 
interview, teachers appear comfortable with the Readers’ and Writers’ Workshops, and despite 
the difficulty of implementing TERC Investigations, are confident that it will yield the results 
they are looking for. 


Specific learning objectives for each lesson are required. The team saw these objectives posted in 
classrooms and heard them reiterated during lessons. Teachers’ plan books were visible, 
complete and up-to-date. In classrooms there were no discipline problems observed; students 
worked quietly and with engagement. In one first grade classroom children easily used words 
like “equation” and were able to define and give examples of compound words. Third grade 
students were observed learning about figurative language including similes and metaphors. Use 
of graphic organizers such as a Venn diagram was observed. One fifth grader was seen reading 
Huckleberry Finn. Student work assignments displayed in classrooms were standards based. 
Examples included personal narratives, persuasive essays, and narratives about the conversion of 
fractions to decimals and percents. 


In an interview with 25 students from grades four and five, they indicated that teachers try to 
help them; that they are practicing how to take tests and reported regular practice using past 
MCAS questions; and that they are reading books that they are enjoying. When queried on what 
they were writing in language arts, these students mentioned personal narratives, fictional stories, 
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cinquains and persuasive essays. When asked to describe what a persuasive essay was, one 
student did so quite articulately. 


C. Is the school receiving adequate guidance and support from the district leadership? 


In general the school is receiving adequate guidance from the district leadership although fiscal 
support is not ideal. For example, the school does not have sufficient funds to provide both a 
computer lab and a school library. Some teachers reported that they need additional books and 
the computers in the classrooms are not up-to-date. The school had to give up an art teacher in 
order to expand library services. However, class size is reasonable and in some cases is less than 
the 22-25 maximum allowed. The Superintendent expressed his strong support for the Principal 
and his belief that the school can improve. He was knowledgeable about the school’s situation 
and told the team that he had supervised the Principal directly for two years previously. The 
Deputy Superintendent in charge of the cluster for the John Marshall Elementary School 
approved the school’s WSIP. 


As indicated above, there are a number of district programs housed at the school that provide a 
challenge to staff, such as the language cluster, the LAB cluster, and the ELL program that 
provides services to 120 students who are at step zero in English language acquisition. In 
addition, approximately ten third grade students leave each year for the advanced work program. 
The Panel Review Team could not determine the effect of these populations on whole school 
achievement or upon subgroups. This may also raise a question about equity or adequacy of 
district support for resources needed to enable this diverse student body to attain proficiency. 


The district offered training in data analysis and root causes as part of the development of the 
WSIP and provided a consultant from the Harvard School of Education to provide training for 
the principals. In addition, a Harvard intern was assigned to the John Marshall Elementary 
School and in return the principal and two other members of the ILT were able to take a graduate 
class on data analysis at Harvard. 


Two years ago, the district mandated the introduction of the math TERC Investigations program 
but was not able to provide sufficient math coaching for teachers to implement the program. 
Three of six teachers in grade four are new and lack training in TERC, yet the coach comes only 
twice a month to help them. According to the Superintendent, the district has been unable to 
locate trained math specialists to hire for coaching positions. There appears to be sufficient 
coaching available for implementation of Readers’ and Writers’ Workshops. 


The district has provided a math vocabulary list by unit, in addition to the pacing guide. This 
year the district has begun working on the alignment of the formative assessments with the 
curriculum and MCAS. The district’s MyBPS website offers a variety of supports for teachers to 
download, including strategies and curriculum guides. The Superintendent told the team that he 
expected the formative assessments to be posted on the website as well. The school is also a pilot 
site for the Boston Plan for Excellence formative assessment development. 
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CONCLUSION 


The Panel Review Team concluded that the John Marshall Elementary School has a sound 
school improvement plan for improving student achievement. The school has analyzed and 
identified student instructional needs and is implementing strategies designed to move students 
forward. Conditions appear to be in place at the school to support successful implementation of 
the plan. All stakeholders have a common understanding of the plan and its strategies and share a 
focused vision. The staff, operating under the leadership of a strong, respected principal who is 
valued by district leadership, is cohesive, committed and determined to succeed. 


Conditions that may provide stumbling blocks along the way result from a lack of district 
provided resources, previously identified, such as deficits in available math coaches or funds to 
provide a full complement of services like a computer lab and a school library. The number of 
relatively new teachers in the building (14), teachers working outside their areas of certification, 
chronically absent students, the fact that after school professional development cannot be 
mandated above the contractual 30 hours, and lack of parental involvement through a PTA or 
PTO were also noted by the Team as potential obstacles to success. 
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APPENDIX A 
Team Members 


Nick Feldman, Panel Chairperson, is a consultant for SchoolWorks. 


Zita Samuels, Panel Co-Chair, is a professional grant writer and evaluator and a consultant for 
SchoolWorks 


Denise Delorey, PHD, Panel Coordinator, Massachusetts Department of Education, Office of 
Accountability and Targeted Assistance. 


Dr. Kenneth Cooper has been principal of the Doyon Memorial School, Ipswich Public 
Schools, for the past seventeen years. 


Susan Cote is assistant superintendent/Title I director in the Swansea Public Schools. 
Joanne Garska is an evaluation team leader at the Boland School, Springfield Public Schools. 


Susan Nutting is principal of the Thomas Nash Primary School, Weymouth Public Schools. 
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APPENDIX B 


UNDER-PERFORMING PANEL REVIEW SCHEDULE 
Detailed Schedule for Review Panel School Site Visit 


The times specified on the following schedule may be adjusted slightly to align with the 
daily schedule and practices in each of the schools being reviewed. 


Day 1 

10:30—12:00 Panel chairperson and panel coordinator meet to discuss and clarify roles, prepare 
for the first team meeting, and review general logistics/schedule for the review. 
[location: hotel] 

12:00—2:00 p.m. Team meeting # 1: team meets for the first time to discuss each panelist’s 
individual analysis; team forms preliminary judgments on key questions. 
[location: hotel] 

2:00—3:00 p.m. Panelists meet with the district Superintendent (and Assistant Superintendent, if 
appropriate). [location: hotel] 

3:30—4:30 p.m. Panelists meet with Principal (and one other school-based individual, if 
appropriate). [location: the school] 

6:00—S&:00 p.m. Team meeting # 2: panelists synthesize interview information, further define 
findings, prepare questions, and develop a team strategy for Day 2 of the review. 
[location: hotel] 

Day 2 
All activities take place in the school 

7:30—8:00 a.m. Panelists meet with the Principal 

$:00—S&:30 a.m. Panelists meet with the School Council 

8:30—9:00 a.m. Panelists meet with Focus Groups. The Panel Review Coordinator and the 


Principal will identify participants for each Focus Group. The groups will be 
organized, as appropriate, to include groups of individuals who can respond to 
questions designed for parents, students, classroom teachers, curriculum 
facilitators, content-area specialists, grade-level instructors, or other specific 


inquiry groups. 


Panelist E& F 
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9:00—11:00 a.m. 


|__| Panelist A_| Panelist | Panelist C 


11 a.m.—12:30 p. 


12:30—1:00 p.m. 


1:00—2:00 p.m. 


eel al 


1:00-1:30 


1:30-2:00 


2:00—2:30 p.m. 


2:30—5:00 p.m. 


February 17, 2004 


9-10 a.m. Observe Observe Observe Observe Observe Observe Observe 
teacher 1 teacher 3 teacher 5 teacher 7 teacher 9 teacher 11 teacher 13 
and teacher | and teacher | and teacher | and teacher | and teacher | and teacher | and teacher 


2 
Interview 
teacher 1 


and teacher | and teacher | and teacher | and teacher | and teacher | and teacher | and teacher 
2 4 6 8 10 12 14 
individually | individually | individually | individually | individuall individuall individually 


Classroom observations and teacher interviews* 


4 
Interview 
teacher 3 


6 
Interview 
teacher 5 


10 
Interview 
teacher 9 


8 
Interview 
teacher 7 


12 
Interview 
teacher 11 


14 
Interview 
teacher 13 


m. Team meeting # 3: panelists meet to discuss findings so far and to plan the 
remainder of the day (working lunch) 


Panelists use time as needed to analyze findings and to gather more information; 
panelists are encouraged to roam the entire school and visit classrooms not yet 
seen. 


Panelists meet with teachers in groups*; consultant co-chair is free to work on 


report 
& G 


Interviews Interviews Interviews Prepare 
or Focus or Focus or Focus or Focus or Focus report 
Groups Groups Groups Groups Groups 
Interviews Interviews Interviews Interviews Interviews 

or Focus or Focus or Focus or Focus or Focus 
Groups Groups Groups Groups Groups 


Interviews Interviews 


Closing meeting with the principal to discuss next steps (all panelists are present) 


Team meeting # 4: panelists deliberate and form conclusions 


Massachusetts Department of Education 


BOSTON PUBLIC LIBRARY 


| 
3 


